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CFO 12836 US 



IMAGE PROCESSING APPARATUS, METHOD 



AND RECORDING MEDIUM THEREFOR 



10 



BACKGROUND OF THE INVENTION 
Field of the Invention 

The present invention relates to an image 
processing apparatus capable of estimating and 
correcting color data which vary according to the 
illuminating light, and a method and a recording medium 
therefor. 

Related Background Art 

Utilizing the spectral reflectance R(X) of a 
reflective object, the spectral distribution P{X) of 
the illuminating light and the isochromatic functions: 

x(?^),y(>^),z(>^) A 
the colorimetric values (X, Y, Z; three stimulation 
values) of the reflective object under certain 
illuminating light can be represented by: 



r R(X)-P(X)*x(X)dX 



Jvis 



y=Jc 



B 



Jvis 



Jvis 



wherein the constant k is represented by: 




C 
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and the integration: 

t.) 

is executed within the visible wavelength range. 
Consequently the colorimetric values of the 
5 reflective object vary according to the variation of 
the illuminating light. Also according to the 
definitions described above, the spectral distribution 
P(k) of the illuminating light and the spectral 
reflectance distribution R{X) are required in order to 
10 obtain the exact colorimetric values in such state. 

For this reason, the colorimetric values of the 
reflective object under arbitrary illuminating light 
have conventionally been determined from the spectral 
reflectance R(>^) of the reflective object and the 
15 spectral distribution P(X) of the illuminating light. 

The above-mentioned method can be easily executed 
and can determine the exact colorimetric values in case 
the reflective object consists of areas of several 
colors (spectral reflectance R(X)). On the other hand, 
20 in case the reflective object is for example an image, 
the object in general has color information in each of 
finely divided many areas (pixels). Consequently there 
is required a large memory capacity for storing 
spectral reflectance R(?^) for each pixel, and the 
25 information ordinarily held for each pixel is the 
chromaticity values (X, Y, Z) under a specified 
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condition (specifying illuminating light or 
colorimetric method) or equivalent RGB chromaticity 
values. In order to determine the colorimetric values 
of the reflective object for the arbitrary illuminating 
5 light in the above-described method, there is required 
the spectral reflectance R{X) for each pixel, so that 
the spectral reflectance R(>^) is determined again from 
the information corresponding to the aforementioned 
colorimetric values (X, Y, Z) for each pixel or the 
10 spectral reflectance R(>^) is measured again for each 
pixel . 

In case the information obtained for each pixel is 
the values corresponding to the colorimetric values 
(such as the aforementioned XYZ values of RGB values) 

15 under a specified condition, the colorimetric values of 
the reflective object under arbitrary illuminating 
light can be determined, in addition to the above- 
described method according to the foregoing definitions 
based on the spectral reflectance R(X) of the object 

20 and the spectral distribution F{\) of the illuminating 
light, by a method of directly converting the 
information for each pixel, corresponding to the 
colorimetric values under the specified condition, into 
the colorimetric values under the arbitrary 

25 illuminating light utilizing a matrix, a 

three-dimensional look-up table or a neural network. 
The conversion function (above-mentioned matrix, three- 
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dimensional look-up table or neural network) is 
determined for each of the required plural illuminating 
lights- 

As explained in the foregoing, the color imetric 
5 values of the reflective object vary according the 
change in the illuminating light. In an image 
reproducing process or the like, there are often 
required the colorimetric values of the reflective 
object under arbitrary illuminating light. 
10 In case the information obtained for each pixel is 

the values corresponding to the colorimetric values 
(such as the aforementioned XYZ values of RGB values) 
under a specified condition, the colorimetric values of 
the reflective object under arbitrary illuminating 
15 light can be determined by a method of directly 
converting the information for each pixel, 
corresponding to the colorimetric values under the 
specified condition, into the colorimetric values under 
the arbitrary illuminating light utilizing a matrix, a 
20 three-dimensional look-up table or a neural network. 

The conversion function (above-mentioned matrix, three- 
dimensional look-up table or neural network) is 
determined for each of the required plural illuminating 
lights. 

25 The above-described method is acceptable in case 

the number of the required illuminating lights is 
limited. However, for example in the ordinary office 
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environment, the condition of lighting changes in 
various manner according to the kind of the 
illuminating light source, the time-dependent change 
thereof, and the change in the state of the incoming 
5 external light such as the solar light, and it is 

difficult to prepare or store in advance the conversion 
functions required corresponding to such changes. 

As explained in the foregoing, in order to 
determine the colorimetric values under arbitrary 
10 illuminating light for an image or the like that 

requires information for each of a large number of 
pixels, the conventional methods have been associated 
with drawbacks of requiring a large amount of 
information such as preparing the spectral reflectance 
15 for each pixel or preparing a large number of 
conversion functions corresponding to various 
illuminating light conditions. 

On the other hand, owing to the recent 
commercialization of various color image processing 
20 equipment, the color images can be handily processed 

not only in the special fields such as designing based 
on the computer graphics but also in the ordinary 
offices. It has however been difficult to consider the 
color of the printout on the monitor, because the color 
25 of the image prepared on the monitor does not in 

general match that of the printout obtained from the 
printer. In order to solve such drawback, there has 
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been considered and proposed the color management 
system. 

The color management system is to cancel the 
difference in color between the different devices by 
5 employing a common color space. This system is based 
on a principle that a color described by same 
coordinate values in a same color space should always 
look same and is to match the apparent color by 
representing all the colors in a common color space and 
10 matching the coordinate values. One of the currently 
employed methods for canceling the difference between 
the devices employs the CIE-XYZ color space and 
utilizes the XYZ stimulation values which are the 
internal description coordinates therein. 
15 However such method may still be insufficient in 

case the media used for reproduction are different, for 
example the image on a monitor and the printed image. 
Fig. 27 shows the environment for observing an image on 
a monitor and a printout. In the following it is 
20 assumed that an image 1202 same as that on a print 1201 
is displayed on a monitor 1203. 

The printed image or the image displayed on the 
monitor is not observed under constant ambient light, 
but the ambient light 1204 shown in Fig. 27 changes by 
25 the opening or closing of a window or the replacement 

of the illuminating light source, and the image appears 
differently by such change. Consequently, even when 
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isochromaticity can be obtained under certain ambient 
light, it cannot be preserved even under the same 
conditions in case the ambient lighting condition is 
altered. 

5 Though the foregoing consideration has been 

explained in the comparison of the printed image and 
the image displayed on the monitor, it is generally 
applicable to the comparison of a color presentation by 
reflecting illuminating light and a light-emitting 

10 color presentation. More specifically, such phenomenon 
occurs also in case of taking an object such as a 
person or a sculpture and displaying it on a monitor or 
presenting as a transmissive display. 

As explained in the foregoing, the appearance of 

15 the image varies according the change in the image 
observing environment. In the conventional art 
employing different media, the images providing 
isochromaticity under a certain situation no longer 
provide isochromatic feeling by the change in the image 

20 observing environment. 

SUMMARY OF THE INVENTION 

An object of the present invention is to obtain 
data under desired illumination in a simple manner with 
25 a limited memory capacity. 

The above-mentioned object can be attained, 
according to the present invention, by an image 
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processing method for converting data dependent on a 
first illuminating light into data dependent on a 
second illuminating light, the method comprising steps 
of storing conversion data for plural illuminating 
5 lights of different characteristics, generating data 
corresponding to the second illuminating light and 
indicating the proportion of synthesis of the plural 
illuminating lights of the different characteristics, 
and converting the data dependent on the first 
10 illuminating light into those dependent on the second 

illuminating light, based on the conversion data of the 
plural illuminating lights of different characteristics 
and on the data indicating the proportion of synthesis. 
Another object of the present invention is to 
15 compensate the appearance of the color of the displayed 
image regardless of the observing environment. 

Still another object of the present invention is 
to provide a user interface allowing easy setting of 
the correction for the ambient light, 
20 The above-mentioned objects can be attained, 

according to the present invention, by an image 
processing method comprising steps of: 

setting a coefficient indicating the ambient 
illuminating light characteristics according a manual 
25 instruction; 

entering image data dependent on an input device; 

and 



effecting correction for the ambient light on the 
entered image data based on the input device, a display 
device and the ambient illuminating light 
characteristics coefficient, thereby achieving 
conversion to image data dependent on the display 
device . 

Still other objects of the present invention, and 
the features thereof, will become fully apparent from 
the following description which is to be taken in 
conjunction with the attached drawings • 

BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a chart showing spectral characteristics 
of light sources of different spectral distributions; 

Fig. 2 is a chart showing examples of colorimetric 
values (a*, b*) of a same reflective object illuminated 
by the light sources of different spectral distribution 
characteristics ; 

Figs. 3, 4 and 5 are charts showing examples of 
the result obtained by the algorithm of the present 
invention; 

Fig. 6 is a view showing an example of the 
colorimetric value estimating system; 

Fig. 7 is a view of a test chart consisting of 77 
color patches, employed for determining the 
illuminating light characteristic matrix; 

Figs- 8, 9, 10 and 11 are views showing examples 
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of the colorimetric value estimating systems- 
Fig. 12 is a chart showing the spectral 

sensitivity of a sensor in the illuminating light 

sensing unit; 

Fig, 13 is a view showing an example of the 

colorimetric value estimating system; 

Fig. 14 is a view showing an example of the user 

interface for designating the illuminating light; 

Fig. 15 is a block diagram showing a color image 

display system of an embodiment 4; 

Fig. 16 is a view showing the flow of data and 

results of calculations in the calculation of a matrix 

MTX,^; 

Fig. 17 is a view showing a user interface image 
frame; 

Fig. 18 is a flow chart showing the operations in 
an ordinary operation mode; 

Fig. 19 is a flow chart showing the operations in 
a user interface mode; 

Fig. 20 is a flow chart showing the operations in 
the execution of the user interface mode; 

Fig. 21 is a block diagram showing a color image 
display system of an embodiment 5; 

Fig. 22 is a view showing the user interface image 
frame of the embodiment 5; 

Figs. 23 and 24 are flow charts showing the 
operation in the user interface mode of the embodiment 



Fig* 25 is a flow chart showing the operations in 
the execution of the user interface mode in an 
embodiment 6 ; 

Fig. 26 is a block diagram showing a color image 
display system of a variation of the embodiment 1; and 

Fig. 27 is a view showing an image observing 
environment . 

DESCRIPTION OF THE PREFERRED EMBODIMENTS 
[ Embodiment 1 ] 

At first there will be explained the principle of 
estimating the colorimetric values of the reflective 
object,. to be employed in the following embodiments. 
For converting three stimulation values under certain 
illuminating light into those under another 
illuminating light, there is known, for example, a 
conversion method corresponding to the change in the 
color temperature, such as the color adaptation 
conversion (such as the method of von Kries). However, 
as often felt in the image observation under the 
natural daylight and under the daylight type 
fluorescent lamp, certain colors may be felt 
differently even if the color temperature is same. 
Also certain colors may be felt different even if the 
isochromaticity is attained by the color adaptation 
conversion for the colors close to the achromatic 
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color. Such phenomena are assumed to result from the 
spectral distribution characteristics of the 
illuminating light, and the evaluating method therefor 
is already known, for example the method of evaluating 
the color rendering of the light source according to 
jIS-Z-8726 (1990)- Fig, 1 shows examples of the light 
sources substantially same in the color temperature but 
different in the spectral distribution. In Fig. 1, a 
curve 11 indicates the spectral distribution of a 
white, ordinary fluorescent lamp while a curve 12 
indicates that of a white, high color rendering 
fluorescent lamp. Fig. 2 shows the colorimetric values 
of a same reflective object, illuminated by the two 
light sources shown in Fig. 1. In Fig. 2, square marks 
21 indicate the colorimetric values corresponding to 
the illuminating light of the spectral characteristics 
12 in Fig. 1, and the ends 22 of line segments 
corresponding to that of the characteristics 11 in Fig. 
1. 

The method for estimating the colorimetric values 
of the reflective object to be employed in the 
following embodiments serves to estimate and correct 
the difference of color, resulting from the difference 
in the spectral distribution characteristics of the 
illuminating light . 

In this method, a lighting characteristic matrix 
CR, defined by the following equation: 



- 13 - 



CR = IHi,3»CRH,+ (l-IHi,J*CR^,i E 
is utilized as means for estimating and correcting the 
difference of color, resulting from the difference in 
the spectral distribution characteristics of the 
illuminating light . 

In the above-mentioned equation, CR^^ is a lighting 
characteristic matrix corresponding to the natural 
daylight, the incandescent lamp or the illuminating 
light, obtained from a standard light source defined in 
the Japan Industrial Standards or a light source of 
satisfactory color rendering such as a high color- 
rendering fluorescent lamp. 

CR^i is a lighting characteristic matrix 
corresponding to the illuminating light obtained from a 
light source of low color rendering, as defined in the 
Japan Industrial Standards, such as an ordinary 
fluorescent lamp. 

Also IHks is a lighting characteristic coefficient, 
assuming a value between 0 and 1. 

In case IHj^^ = 1, CR coincides with CR^^. The 
matrix obtained in this case corresponds to the 
illuminating light of satisfactory color rendering, 
such as the natural daylight, the incandescent lamp, 
the JIS defined standard light source of the JIS 
defined high color rendering fluorescent lamp. 

In case IHj^^ = 0, CR coincides with CR^^i- In this 
case, the matrix corresponds the illuminating light of 
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low color rendering, such as the JIS defined ordinary 
fluorescent lamp. 

In case 0 < IH^^ < 1, the lighting characteristic 
matrix CR corresponds to the illuminating light 
obtained by mixing CRj,^ and CR^i with a mixing ratio IHi^^- 
In this manner it is rendered possible to generate 
the lighting characteristic matrix corresponding to the 
illuminating lights of various color rendering 
properties, based on the lighting characteristic 
matrixes corresponding to the illuminating lights of 
different color rendering properties. 

Figs. 3, 4 and 5 show the results of estimation of 
the colorimetric values under arbitrary illuminating 
light, utilizing those under a certain specified 
condition. In the illustrated cases, the high color 
rendering illumination of daylight color is used as the 
specified condition. As the arbitrary illuminating 
light, the case shown in Fig. 3 employs an ordinary 
white fluorescent lamp (light source of low color 
rendering), while the case in Fig. 4 employs a high 
color rendering white fluorescent lamp (light source of 
high color rendering), and the case in Fig. 5 employs a 
daylight white fluorescent lamp of three-wavelength 
type (light source of medium color rendering). In 
these drawings, square marks indicate the colorimetric 
values of pixels under the above-mentioned specified 
condition, while crosses indicate the actually measured 
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color imetric values under the arbitrary illuminating 
light and triangles indicates the estimated 
colorimetric values. 

Fig. 6 shows an example of the flow of the 
5 colorimetric value estimating process utilizing the 
estimating method described above. The estimating 
process can be realized through execution of different 
process units shown in Fig. 6, by a CPU according to a 
program stored in a memory medium and adapted to 

10 execute the different process units. 

An image data conversion unit 61 converts the 
input image data XYZ into estimated image data X'Y'Z' 
under an arbitrary light source according to the 
following equation, utilizing the lighting 

15 characteristic matrix CR calculated in an illuminating 
light characteristic matrix calculation unit 62, 
wherein the input pixel data XYZ are normalized to the 
values under a high color rendering light source of a 
color temperature of 6500": 









X 


20 


y' 


= CR 


y 




z' 




z 



The lighting characteristic matrix CR is 
calculated, in the lighting characteristic matrix 
calculation unit 62, according to the following 
equation: 



CR = IHi,,»CRj,,+ (l-IHi,3)»CRj,, G 
based on the lighting characteristic matrix CR^^ stored 
in a standard lighting characteristic matrix storing 
unit 63 and corresponding to the illuminating light 
obtained from the light source of high color rendering 
such as the natural daylight, incandescent lamp, JIS- 
defined standard light source or JlS-defined high color 
rendering fluorescent lamp, the lighting characteristic 
matrix CR^i corresponding to the illuminating light 
obtained from the light source low color rendering such 
as the JlS-defined ordinary fluorescent lamp and a 
lighting characteristic coefficient lU^^ given by a 
lighting characteristic coefficient instructing unit 
64, 

The lighting characteristic matrixes CR^^r CR^i can 
be obtained by determining the three stimulation values 
under the above-mentioned illuminating lights and those 
under the standard light source, for example with a 
test chart containing 77 color patches as shown in Fig. 
7 and executing optimization for example by the 
attenuated minimum square method* 

The lighting characteristic coefficient lU^^ 
indicates information on the mixing ratio of the high 
color-rendering light source and the low color- 
rendering light source as explained in the foregoing, 
and is set by the lighting characteristic coefficient 
instructing unit 64 according to the kind of the 
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designated illuminating light, 

A plurality of the lighting characteristic 
coefficients IHy,^ are prestored in the lighting 
characteristic coefficient instructing unit 64 
corresponding to the kinds of the illuminating lights* 

Fig. 14 shows an example of the user interface 
(UI) relating to the setting of the lighting 
characteristic coefficient. The setting of the present 
embodiment includes, as shown in the user interface 
140, a selection mode 141 and a setting mode 142. 

The selection mode 141 displays the kinds of the 
illuminating lights stored in the lighting 
characteristic coefficient instructing unit 64 in the 
form of a pull-down menu, for selection by the user. 

The setting mode 142 sets the value of the 
lighting characteristic coefficient according to the 
instruction of the user. According to the position of 
a black triangle mark on a bar, selected by the manual 
instruction of the user, the lighting characteristic 
coefficient instructing unit 64 calculates an sets the 
value of the lighting characteristic coefficient. The 
black triangle mark positioned at the left-hand end of 
the bar sets 0 as IHi^^ and that at the right-hand end 
sets 1. Also for setting the lighting characteristic 
coefficient while confirming the effect of the actually 
set lighting characteristic coefficient, there are 
provided a mode 143 for displaying the color patches in 
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a user interface 145 and a mode 144 for displaying the 
original image. 

In case the patch 143 is selected by the user in 
the UI 140, the patch data representing the 
predetermined colors are processed according to the 
lighting characteristic coefficient set on the bar of 
the UI 140 and the obtained X'Y'Z' data are corrected 
according to the monitor profile and displayed on the 
monitor, whereby the user can confirm the effect of the 
set lighting characteristic coefficient. 

Also in case the preview mode 144 is selected by 
the user in the UI 140, the original image read 
according to the instruction of the user is processed 
according to the lighting characteristic coefficient 
set on the bar and displayed as shown in the UI 146, 
whereby the user can confirm the effect. 

The above-described embodiment allows to easily 
estimate and correct the colorimetric values under 

arbitrary light source with a limited memory capacity. 

[ Embodiment 2 ] 

The foregoing embodiment 1 does not execute the 

correction of the color temperature. The present 

embodiment provides the process including the 

correction of the color temperature in the process of 

the embodiment 1 . 

As shown in Fig. 8, a color temperature conversion 

unit 81 is added to the colorimetric value estimation 
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process shown in Fig. 6, whereby the conversion of the 
color temperature can be easily achieved* The color 
temperature conversion unit 81 converts the estimated 
output image data X'Y^Z* into image data X"Y"Z" under 
the illuminating light of a desired color temperature 
according to the following equation: 



'x"' 




'x'- 


y" 


= CR 


y' 


z" 




z< 



for example by the method of Von. Kries utilizing a 
color temperature conversion matrix CT corresponding to 
the information (for example XYZ stimulation values) 
relating to the arbitrary color temperature and given 
from the lighting characteristic coefficient 
instructing unit 64. 

Instead of the method of Von. Kries, there may be 
applied another method of color temperature conversion. 
The method of preparation of the color temperature 
conversion matrix is detailedly explained for example 
in "Color Engineering; published by Publishing Office 
of Tokyo Denki University; Chapter 6 Development of CIE 
color representation system". 

In the process shown in Fig. 8, the image data 
conversion unit 61 and the color temperature conversion 
unit 81 are separately provided, but the process may be 
collectively executed in the image data conversion unit 
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61 as shown in Fig- 9. 

The colorimetric values under desired illumination 
can be easily estimated and corrected, including the 
conversion of color temperature, by synthesizing the 
color temperature conversion matrix CT with the 
lighting characteristic matrix CR, calculated in the 
lighting characteristic matrix calculation unit 62, as 
indicated by the following formula to obtain a lighting 
characteristic matrix CR' including the color 
temperature conversion: 

CR' = CT»CR ^ 
and converting the image data by such matrix CR' • 

It is also possible, as shown in Fig. 10, to 
calculate the color temperature conversion matrix CT in 
the lighting characteristic matrix calculation unit 62 
and to execute the matrix calculation based on the 
obtained result. 

It is also possible to automatically set the 
lighting characteristic coefficient or the color 
temperature information without the instruction of the 
user through the user interface as shown in Fig. 14, by 
providing, as shown in Fig. 11, a light sensing unit 
111 and a lighting characteristic coefficient 
calculation unit 112. 

The light sensing unit 111 is provided with a 
sensor of sensitivity characteristics for example as 
shown in Fig. 12 for measuring the illuminating light. 
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and the BGR output signals of the sensor are processed 
to obtain individually the above-mentioned set values 
(color temperature, lighting characteristic 
coefficient ) . 
[Embodiment 3] 

Though in the foregoing embodiments 1 and 2, the 
lighting characteristic matrix is employed for 
estimating and correcting the difference in color 
resulting from the difference in the spectral 
distribution of the illuminating lights, the present 
invention is not limited to a method utilizing such 
matrix and can also be attained for example with a 
three-dimensional look-up table. 

In this case, for estimating and correcting the 
difference in color resulting from the difference in 
the spectral distribution of the illuminating lights, 
there is employed a lighting characteristic three- 
dimensional look-up table CR3LUT, which is defined by 
the following equation: 

CR3LUT = IHi,3» CR3LUTj,,+ ( l-IH^.^ ) • CR3LUTj,i J 
wherein CR3LUTj,r is a lighting characteristic three- 
dimensional look-up table corresponding to the natural 
daylight, the incandescent lamp or the illuminating 
light, obtained from a standard light source defined in 
the Japan Industrial Standards or a light source of 
satisfactory color rendering such as a high color- 
rendering fluorescent lamp. 
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CR3LUTj,i is a lighting characteristic three- 
dimensional look-up table corresponding to the 
illuminating light obtained from a light source of low 
color rendering, as defined in the Japan Industrial 
Standards, such as an ordinary fluorescent lamp. 

Also IHi^3 is a lighting characteristic coefficient, 
equivalent to that in the foregoing embodiments. 

Fig, 13 shows an example of the colorimetric value 
estimating system utilizing such colorimetric value 
estimating method, wherein an image data conversion 
unit 131 converts the input image data XYZ into 
estimated image data X'Y'Z' under a designated light 
source according to the following equation: 

(X^ Y', Z') = CR3LUT (X, Y, Z) K 
utilizing the lighting characteristic three-dimensional 
look-up table CR3LUT calculated in the lighting 
characteristic three-dimensional look-up table 
calculation unit 132. 

It is also possible to estimate and correct the 
difference in color resulting from the difference in 
the spectral distribution of the illuminating lights, 
employing a function as more general means, and, in 
such case, the arbitrary illuminating light 
characteristics are defined by a function (for example 
CRF). The function CRF satisfies the following 
relationship : 

CRF=CRF (CRF^,, CRF^,,, IK^J L 
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which may assume a specific form as in the foregoing 
embodiments : 

CRF = IH^3^ CRF^,+ ( J • CRF^^ M 

or 

CRF= IHi,,^* CRFj,,+ ( 1 - IH^ J CRF^, N 

or 

CRF= IHt,,2. CRFj,^+ ( 1 -IK^J ) • CRF^, O 
wherein CRF^^ is a lighting characteristic function 
corresponding to the natural daylight, the incandescent 
lamp or the illuminating light, obtained from a 
standard light source defined in JIS or a light source 
of satisfactory color rendering such as a high color- 
rendering fluorescent lamp; and 

CRF^i is a lighting characteristic function 
corresponding to the illuminating light obtained from a 
light source of low color rendering, as defined in JIS, 
such as an ordinary fluorescent lamp. 

Based on the above-described function, the 
colorimetric values XYZ are converted into the values 
X'Y'Z' as indicated by the following equation: 

(X' , Y' , Z' ) = CRF (X, Y, Z) P 
wherein IH^^ is the lighting characteristic coefficient 
equivalent to that in the foregoing embodiments. 
[Embodiment 4] 

Fig, 15 shows an example of the color image 
display system constituting an embodiment 4 of the 
present invention. 
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The upper half in Fig, 15 principally shows the 
flow of the image data and the signal processing, while 
the lower half is composed of data storing units and 
data calculating units for the signal processing. 

An image data input unit 1101 is connected with a 
scanner, a digital camera or an image memory device and 
receives the image of an object, a printed matter etc. 
in the form of color image signals RinGi^Bi^. 

The image signals Ri^Gi^Bi^ entered by the image data 
input unit 1101 are subjected, in an input gamma 
conversion unit 1102, to the correction of the gamma 
characteristics of the input device providing these 
image signals, utilizing a look-up table, and are 
converted into in- system input signals Rin'^i^'^.^' as 
indicated by the following equations: 



Ri„' 


= LUT,i„ 


(Rin) 




Gin' 


= LUT„,„ 


(Gin) 


Q 


Bin' 


= LUT3,„ 


(Bin) 





The above-mentioned gamma conversion LUT is stored 
in an input device characteristic data storing unit 
1106, and is set in the input gamma conversion unit 
1102 by a system control unit 1111 according to the 
information from a user interface unit 1110. 

The in-system input signals Rin'^i^'^.^' are 
subjected to matrix conversion represented by the 
following equation in an image conversion unit 1103, 
thereby being converted into in-system output signals 
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= MTX, 



gh 



G- ' 
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A conversion matrix MTX^^ is determined in an image 
conversion matrix calculating unit 1109 according to 
the instruction from the system control unit 1111 as 
will be explained later, utilizing the characteristics 
of the image input device, image display device and 
observing ambient light obtained from the user 
interface unit 1110, the input device characteristic 
data storing unit 1106, the display device 
characteristic data storing unit 1107 and the 
conversion matrix calculation data storing unit 1108, 
and is set in the image conversion unit 1103. Fig. 16 
shows the flow of various data and the results of 
calculations in the above-described operations. 

The in-system output signals Rout ' G^ut ' B^^t ' 
subjected, in an output gamma conversion unit 1104, to 
a correction according to the gamma characteristics of 
the display device on which these signals are to be 
displayed, thus being converted into output image 
signals RoutGoutBout • This conversion is achieved by a 
look-up table conversion utilizing the signals 
Rout ' G^ut ' B^ut ' SIS represented by the following equation: 



Rout = 




(Rout') 




Gout = 






S 


Bout ' 




(Bo„.' ) 





The above-mentioned gamma conversion LUT is stored 
in the display device characteristic data storing unit 
1107, and is set in the output gamma conversion unit 
1104 by the system control unit 1111 according to the 
information from the user interface unit 1110. 

The image display unit 1105 is composed of a 
monitor such as a cathode ray tube or a liquid crystal 
display device and displays an image according to the 
output image signals Rout^out^out- 

In the following there will be explained the 
function of the system control unit 1111. 

The system control unit 1111 controls the 
functions of the present system. 

Also the system control unit 1111 selects, through 
the user interface shown in Fig. 17, a gamma conversion 
look-up table matching the input device and a gamma 
conversion look-up table matching the output device 
respectively from the input device characteristic data 
storing unit 1106 and the display device characteristic 
data storing unit 1107, and sets these tables 
respectively in the input gamma conversion unit 1102 
and the output gamma conversion unit 1104. 

Also as shown in Fig. 16, the system control unit 
1111 sends, through the user interface shown in Fig. 
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17, following data to the image conversion matrix 
calculation unit 1109, referring to the input device 
characteristic data storing unit 1106, the display 
device characteristic data storing unit 1107 and the 
conversion matrix calculation data storing unit 1108. 
The above-mentioned data contain an ambient lighting 
color temperature CCT^^^, an ambient light luminosity 
IHj^^, a conversion matrix M^^ corresponding to the input 
device (for converting the in-system input signals 
R 'G. 'B ' into the in-system conversion signals XYZ ) , a 

in m in 

conversion matrix M^^^ corresponding to the display 
device (for converting the in-system conversion signals 
XYZ into the in-system output signals Rout ' ^out ' ^out * ) / 
three stimulation values X^^^jY^i^i^mn of the displayed 
white color of the display device, an ambient light 
characteristic correcting matrix CRh^ corresponding to 
the light source of high color rendering, an ambient 
light characteristic correcting matrix CR^i 
corresponding to the light source of low color 
rendering, an adaptation ratio s, a matrix for 
converting the three stimulation values XYZ of the 
CIEXYZ into visual RGB values, and a calculation 
coefficient . 

The system control unit causes the image 
conversion matrix calculation unit 1109 to calculate 
the image conversion matrix MTX^t, and sets the obtained 
image conversion matrix MTX^^ in the image conversion 
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unit 1103. 

The system control data storing unit 1112 stores a 
flow chart relating to the operations of the system 
control unit 1111 and a user interface image, according 
to which the present system functions. 

The conversion matrix calculation data storing 
unit 1108 stores calculation coefficients and a 
lighting characteristic correcting matrix required for 
calculating the image conversion matrix. 

The image conversion matrix calculating unit 1109 
calculates the image conversion matrix MTX^^ according 
to the instruction of the system control unit 1111 and 
based on the above-mentioned data. 

Fig. 16 shows the flow of various data and of 
results of calculations in the calculation of the image 
conversion matrix MTX^^. 

The image conversion matrix MTX^^ is obtained as 
the product of plural matrixes as indicated by the 
following equation: 

MTXg^=M,,,* M^-^* D* M^- CR* M^, T 
wherein M^^^ is a matrix for converting, for the display 
device, the in-system conversion signals XYZ into the 
in-system output signals Rout ' G^ut ' B,,, ' , whereby the in- 
system conversion signals XYZ not dependent on the 
characteristics of the display device are converted 
into the in-system output signals Rout ' ^out ' B,^t ' dependent 
on (matching) the characteristics of the display 
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device. 

is a matrix for converting, for the input 
device, the in-system input signals ^in' "^m' into the 
in-system conversion signals XYZ, whereby the in-system 
input signals R^JG^^'B-^' dependent on the 
characteristics of the display device are converted 
into the in-system conversion signals XYZ not dependent 
on the characteristics of the device and the signal 
conversion in the system can be rendered independent of 
the individual input, output and display devices. 

is a matrix for converting the three stimulation 

h 

values XYZ of the CIEXYZ color representation system 
employed as the in-system conversion signal system in 
the present invention into responses RhGh^h (visual RGB 
values) at the level of receptor of the human eyes (see 
Foundation of Color Engineering: Asakura Shobo; p,216), 
whereby the image signals can be matched with the human 
characteristics and there can be achieved signal 
processing simulating various processes executed by the 
human at the image observation. 

CR is a matrix for converting the three 
stimulation values X^es^^^^^Z^^s under the illumination with 
the standard light source (D65) into three stimulation 
values X^^Y^s^i^s under the ambient lighting. For 
converting three stimulation values under certain 
illuminating light into those under another 
illuminating light, there is known, for example, a 
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conversion method corresponding to the change in the 
color temperature, such as the color adaptation 
conversion (such as the method of Von. Kries ) * 
However, as often felt in the image observation under 
the natural daylight and under the daylight type 
fluorescent lamp, certain colors may be felt 
differently even if the color temperature is same. 
Also certain colors may be felt different even if the 
isochromaticity is attained by the color adaptation 
conversion for the colors to the achromatic color. 
Such phenomena are assumed to result from the spectral 
distribution characteristics of the illuminating light, 
and the evaluating method therefor is already known, 
for example the method of evaluating the color 
rendering of the light source according to JIS-Z-8726 
(1990)- The matrix CR employed in the present 
embodiment is to correct the difference in the apparent 
colors resulting from the difference in the spectral 
distribution of the illuminating light mentioned above. 
In this manner it is rendered possible to correct the 
difference in the spectral characteristics of the 
ambient illuminating light and to attain better 
isochromatic feeling - 

The ambient lighting characteristic correcting 
matrix CR is desirably determined for each ambient 
illuminating light corresponding to the actual ambient 
situation. For example, this matrix can be obtained by 



determining the three stimulation values under the 
above-mentioned illuminating lights and those under the 
standard light source, for example with a test chart 
containing 77 color patches as shown in Fig. 7 and 
executing optimization for example by the attenuated 
minimum square method. The above-described method can 
be easily executed in case the number of the required 
illuminating lights is limited to several kinds. In 
practice, however, the condition of lighting changes in 
various manner according to the kind of the 
illuminating light source, the time-dependent change 
thereof, and the change in the state of the incoming 
external light such as the solar light, and it is 
difficult, in the above-described method, to determine 
the ambient lighting characteristic correcting matrix 
corresponding to the ambient illuminating light varying 
in various manners . 

On the other hand, in the present embodiment, the 
ambient lighting characteristic correcting matrix CR is 
determined by the following equation: 

CR = IH^^^CRy^, + (1 - IH^,3)*CR^i U 
In the above-mentioned equation, CR^^ is an ambient 
lighting characteristic correcting matrix corresponding 
to the ambient illuminating light obtained from the 
natural daylight, an incandescent lamp or a standard 
light source defined in the Japan Industrial Standards 
or a light source of satisfactory color rendering such 
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as a high color-rendering fluorescent lamp. CR^j, 
employed in the present embodiment is determined by an 
isochromatic feeling experiment conducted with the 
algorithm of the present invention under respectively 
light source, utilizing the ambient lighting 
characteristic correcting matrixes obtained for the 
above-mentioned light sources. In the above-mentioned 
experiment, the corrections with the ambient lighting 
characteristic correcting matrixes corresponding to the 
above-mentioned light sources provided satisfactory 
results in any of the above-mentioned light sources. 
Consequently the light sources corresponding to the 
above-described class can be collectively dealt with, 
and a single correcting matrix can be commonly adopted 
for this class. 

CRhi is an ambient lighting characteristic 
correcting matrix corresponding to the illuminating 
light obtained from a light source of low color 
rendering, as defined in the Japanese Industrial 
Standards, such as an ordinary fluorescent lamp. CR^i 
employed in the present embodiment is determined by an 
isochromatic feeling experiment conducted with the 
algorithm of the present invention under respectively 
light source, utilizing the ambient lighting 
characteristic correcting matrixes obtained for the 
above-mentioned light sources. Also in this case, the 
corrections with the ambient lighting characteristic 
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correcting matrixes corresponding to the above- 
mentioned light sources provided satisfactory results 
in any of the above-mentioned light sources- 
Consequently the light sources corresponding to the 
above-described class can be collectively dealt with, 
and a single correcting matrix can be commonly adopted 
for this class. 

Also lHi^3 is an ambient lighting characteristic 
coefficient entered by the user through the user 
interface unit 1110, and assumes a value between 0 and 
1, 

In case IH^^ = 0, CR coincides with CR^^, The 
matrix in this case corresponds to the illuminating 
light of satisfactory color rendering, such as the 
natural daylight, the incandescent lamp, the JIS 
defined standard light source of the JIS defined high 
color rendering fluorescent lamp* 

In case IHi^^ = 1, CR coincides with CR^^i- In this 
case, the matrix corresponds to the illuminating light 
of low color rendering, such as the JIS defined 
ordinary fluorescent lamp. 

In case 0 < IHj,^ < 1, CR becomes a mixture of CR^^ 
and CR^i with a mixing ratio IH^^^, and corresponds to an 
ambient illuminating light obtained by mixing the light 
source of high color rendering and the light source of 
low color rendering with an arbitrary mixing ratio IH^.^. 
The ordinary ambient illuminating light most frequently 
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encountered is in this state. 

The above-explained experiment indicate that the 
light sources of high color rendering and those of low 
color rendering can be respectively grouped, and the 
ordinary state can be considered as a mixed state of 
these two groups* Assuming such state, an isochromatic 
feeling experiment was conducted with the above- 
mentioned algorithm and under various light source, 
employing the ambient lighting characteristic 
correcting matrix CR for each light source. Also in 
this case, the correction under such light sources with 
the ambient lighting characteristic correcting matrix 
corresponding to each of such light source provided 
satisfactory result. 

In this manner, the above-described algorithm 
allows to obtain the ambient lighting characteristic 
correcting matrix corresponding to various ambient 
illuminating lights, utilizing the ambient lighting 
characteristic correcting coefficient lU^^. 

Finally, D is a color adaptation conversion 
diagonal matrix for effecting color adaptation 
conversion from a state adapted to an observing 
environment for observing an object, a printed matter 
etc. to a state adapted for observing an image on the 
display device, and is defined by: 
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D = 



hw 



0 
0 



0 

G, 



0 

^ 0 



B 



hw 



V 



wherein (R\„, G\,, B\J are the visual RGB values of 
white color under the observing environment, while (Rhwi- 
Gu Bu ) are the visual RGB values of standard white 

hw f nw ' 

color in the observation of the display device • These 
visual RGB values can be obtained from the three 
stimulation values XYZ, utilizing the aforementioned 
matrix in the following manner: 



hw 



hw 



B 



hw 



hw 



z„ 



wherein X'„Y'„Z'„ are the three stimulation values of 
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the ambient illuminating light, which can be obtained 
through the user interface unit and determined in the 
following manner, utilizing the ambient lighting color 
temperature CCT^^^ and the ambient light luminance Yj^^. 
At first the colorimetric values (x'„, y'„) are obtained 
from the ambient lighting color temperature CCTi,^ 
according to the following equations: 

/ ^ 4.6070-10^ ^ 2.9678' 10^ ^ 0,09911-10^ ^ 0.244063 
- " CCT,/ CCT^J CCT^s 

y/^ = -3.000- 870* - 0,275 



Then the three stimulation values are obtained 
according to the following equations, utilizing the 
above-mentioned colorimetric values (x'„, y'^) and the 
ambient lighting luminance Y^s: 

^ w~ — r~ 

y w 

= y 

w -^ks 

/ i~-x'w"y^w 

f^f _ W W . y 

^ „ - 7 -^ks 

y w 

On the other hand, X„Y^Z„ are the three stimulation 
values of the standard white color in the observation 
of the display device, and are determined according to 
the following equations, utilizing the three 
stimulation values X'„Y'^Z'„ of the ambient illuminating 
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light and the three stimulation values y^^m^y.m'Z^^iii of 
the white color on the display device, obtained from 
the display device characteristic data storing unit 
107: 

X, = (1-S)*X'„ + s«x_ 

Y„ = (l-s)»Y^ + s»Y_ Z 
Z„ = (l-s)*Z»„ + s»Z^ 

In case of observing an image on the display 
device, the observer is not considered to adapt to the 
displayed image only but to both the displayed image 
and the ambient illuminating light in a certain 
proportion, and the three stimulation values X„Y„Z„ of 
the standard white color can be determined from the 
foregoing equations, wherein s is a parameter 
(adaptation ratio) indicating the proportion of 
adaptation to the white color on the display, namely 
the influence of the displayed white color, relative to 
the observing environmental white on the standard white 
color. The adaptation ratio s varies according to the 
color temperature of the observing ambient light 
environmental light) and the background color of the 
image (background color of displayed image). For 
example, if the background color changes in gray scale 
from black to white, the proportion of adaptation to 
the ambient light becomes larger as the background 
color becomes closer to black. In the present 
embodiment the adaptation ratio s assumes a value 
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within a range from 0.5 to 0.8, corresponding to the 
above-mentioned conditions . 

In the above-explained manner, the image 
conversion matrix calculating unit 1109 calculates the 
5 image conversion matrix MTX^^ utilizing the 

aforementioned data and according to the instruction of 
the system control unit 1111. 

In the following there will be explained the 
functions of the present embodiment. 
10 In the normal operation mode, the present system 

functions in the following manner. 

Fig. 18 is a schematic flow chart of the functions 
of the normal operation mode. 

The image data input unit 1101 acquires the input 
15 image signals Ri^Pirfitn (slOl), and the input gamma 

conversion unit 1102 executes conversion into the in- 
system input signals R * ' i^B ' as explained in the 
foregoing, utilizing the input device gamma 
characteristic correcting look-up table which is set in 
20 advance corresponding to the current system (sl02). 

Then the image conversion unit 1103 converts the in- 
system input signals ^\n^\n^\n into the in-system 
output signals R ' ou^ ' out^ ' out ^ utilizing the image 
conversion matrix which is also set in advance (sl03), 
25 then the output gamma conversion unit 1104 obtains the 
output image signals Rout^out^out explained in the 
foregoing, utilizing the display device gamma 
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characteristic correcting look-up table which is also 
set in advance (sl04) and the obtained signals are 
displayed by the image display unit 1105 (sl05). 

In case of a change in the input device, the 
output device or the external environment such as the 
ambient illuminating light, the present system can 
renew, in responsive manner through the user interface, 
the input device gamma characteristic correcting look- 
up table, the output device gamma characteristic 
correcting look-up table and the image conversion 
matrix, which constitute the in-system functions, 
according to the changed external environment, whereby 
the system can match the apparent colors in the 
displayed image regardless of the change in the 
external environment of the system. 

Fig. 19 is a flow chart showing the operations 
relating to the user interface of the present system. 

When the user interface mode is accessed in the 
present system, a step s202 reads and displays the 
information of the user interface image shown in Fig. 
17. On this user interface image, the user enters the 
input device in an input device column, the display 
device in a display device column, the color 
temperature CCTj^g of the ambient illuminating light in 
an ambient lighting color temperature column, and the 
ambient lighting characteristic coefficient IHi^^ in an 
ambient lighting characteristic coefficient column. In 
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this image, a column with a downward arrow is provided 
in advance with plural selection items, which are 
displayed as a pull-down menu upon selection of the 
arrow mark. A desired item can be entered by selection 
in the pull-down menu. The above-mentioned information 
are fetched in a step s203, A step s204 discriminates 
the selection of an execution button, and, if it is 
selected, the sequence proceeds to a step s205 to 
execute the flow shown in Fig. 18. If the execution 
button is not selected, the sequence proceeds to a step 
s207 for discriminating the selection of an end button. 
If it is selected, the present operation mode is 
terminated. If the end button is not selected, the 
sequence returns to the step s203 . A step s206 
discriminates the completion of operations, and, if 
completed, the sequence returns to the step s203. If 
not completed, the sequence waits in a stand-by state 
in the step s206. The operations in the user interface 
mode of the present system are executed in the above- 
explained manner. 

In the following there will be explained, with 
reference to Fig. 20, the operations in the execution 
of the user interface mode. 

When the execution of the user interface mode is 
selected, the system control unit 1111 reads, in a step 
s301, a program realizing the flow of the user 
interface mode from the system control data storing 
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unit 1112, and initiates operations according to the 
program. At first a step s302 specifies the image 
input device and selects the corresponding input device 
gamma characteristic correcting look-up table from the 
input device characteristic data storing unit 1106 and 
sets the look-up table in the input gamma conversion 
unit 1102. Also the conversion matrix M.^, for 
converting the in-system input signals irfi\rfi\n ir^*to 
the in-system conversion signals XYZ, is selected 
according to the image input device from the input 
device characteristic data storing unit 1106 and is set 
in the image conversion matrix calculating unit 1109. 
Then a step s303 specifies the image display device and 
selects the corresponding display device gamma 
characteristic correcting look-up table from the 
display device characteristic data storing unit 1107 
and sets the look-up table in the output gamma 
conversion unit 1104. Also the conversion matrix M^^^^, 
corresponding to the image display device and adapted 
for converting the in-system conversion signals XYZ 
into the in-system output signals R ' ^^^G ' ^^^^ ' out ^ and the 
three stimulation values X^.^jY^j-iiZ^^^i of the displayed 
white color of the display device are selected from the 
display device characteristic data storing unit 1107 
and are set in the image conversion matrix calculating 
unit 1109. Then a step s304 sets the ambient lighting 
color temperature CCTi^^, the ambient lighting luminance 
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Yi^3 "t^® ambient lighting characteristic coefficient 

IH^^g, obtained through the user interface, in the image 
conversion matrix calculating unit 1109. A step s305 
sets the lighting characteristic correcting matrix and 
various calculation coefficients, required for the 
calculation of the image conversion matrix, from the 
conversion matrix calculation data storing unit 1108 in 
the image conversion matrix calculating unit 1109. 
Then a step s306 calculates the image conversion matrix 
MTXgj, by the aforementioned method, utilizing the data 
set in the image conversion matrix calculating unit 
1109 in the foregoing steps s302, s303, s304 and s305, 
and sets the obtained matrix MTX^^ in the image 
conversion unit 1103. 

Then the image entered by the image data input 
unit 1101 is converted in the input gamma conversion 
unit 1102, the image conversion unit 1103 and the 
output gamma conversion unit 1104, based on the input 
device gamma characteristic correcting look-up table, 
the display device gamma characteristic correcting 
look-up table and the image conversion matrix set in 
the steps s302, s303 and s306, and the converted image 
is displayed on the image display unit 1106. These 
operations are same as those in the foregoing normal 
operation mode. The above-explained operations are 
executed in steps s307, s308, s309, s310 and s311- 

After these operations, the sequence returns to 
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the aforementioned operations relating to the user 
interface. 

The system of the present embodiment, provided 
with the means for obtaining information from the user 
and the image conversion matrix calculating means, 
through the execution of the above-described 
operations, allows to match the apparent colors on the 
object, the print and on the display regardless of the 
external environment such as the input device, the 
display device and the ambient lighting. 

In the foregoing embodiments, the color 
temperature, luminance and color rendering 
characteristic of the ambient illuminating light are 
entered through the user interface unit 1110, but there 
may be provided an ambient light sensing unit 1114 for 
automatically measuring and setting these parameters of 
the ambient illuminating light, as shown in Fig. 26. 
In the configuration shown in Fig. 26, the ambient 
light sensing unit 1114 employs a sensor of the 
sensitivity characteristics shown in Fig. 12, and the 
output signals RGB of this sensor are processed to 
individually obtain the above-mentioned parameters. 
This configuration is substantially same as the 
foregoing configuration except that the above-mentioned 
parameters are obtained not from the user interface 
unit 1110 but from the ambient light sensing unit 1114, 
and the functions are also substantially similar. Also 
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the above-mentioned set values may be entered 
independently from suitable means. Also such 
configuration allows to set the characteristics of the 
ambient illuminating light by measurement. 

Furthermore, the present embodiment is capable of 
satisfactory signal conversion, taking into 
consideration the spectral characteristics of the 
observing environment and the color adaptation 
characteristics of the observer ( influence of both the 
displayed white color and the ambient light white color 
on the standard white color in the observation of the 
displayed image), by executing the image conversion 
based on the spectral characteristics of the ambient 
illuminating light and also executing the color adapted 
conversion based on the standard white color in the 
observation of the displayed image, obtained from the 
ambient illuminating light and the white color of the 
display device. 

The foregoing embodiment employs the theory of von 
Kries in the color adapted conversion, but there may 
also be applied other color adaptation estimating 
theories. Also the present invention is applicable to 
various hardware configurations and sequence processes 
corresponding thereto. Such sequence process may be 
realized into a logic circuit or a software or an 
algorithm within the aforementioned scope of the 
present invention, and the hardware or the apparatus 
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can be realized according to such algorithm • 

The present invention is further applicable to a 
copying apparatus or a printer provided with a 
previewing function, for displaying the image to be 
printed in advance on a monitor. 
[ Embod imen t 5 ] 

In the foregoing embodiment 4, the image 
conversion matrix to be utilized in the image 
conversion unit 1103 is obtained in the user interface 
unit, by the determination in the image conversion 
matrix calculating unit 1109 for each setting of the 
input device, display device, ambient lighting color 
temperature, ambient lighting luminance and ambient 
lighting characteristic coefficient. 

The input device, the display device and the 
ambient illumination may be limited to a certain number 
of combinations. In such case, such combinations are 
finite and the image conversion matrix may be stored 
corresponding to each of such combinations. 

The present embodiment is suitable for such case, 
and is realized by adding an image conversion matrix 
storing unit 1113 to the color image display system of 
the embodiment 4 shown in Fig. 15. 

Fig. 21 shows the color image display system of 
the present embodiment. 

The image conversion matrix storing unit 1113 
stores the matrix, calculated in the image conversion 
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matrix calculating unit 1109, according to the 
instruction of the user interface unit 1110. According 
to the instruction from the user interface unit 1110, 
the system control unit 1111 stores the image 
conversion matrix from the image conversion matrix 
calculating unit 1109 to the image conversion matrix 
storing unit 1113 or setting the matrix therefrom in 
the image conversion unit 1103, 

The functions of the present embodiment are same 
as those of the foregoing embodiment 4, except for the 
addition of a step of storing the calculated image 
conversion matrix and a step of selecting and setting 
the stored image conversion matrix. 

Fig. 23 is a flow chart of the operations relating 
to the user interface of the present embodiment. 

Also Fig. 22 shows the user interface image of the 
present embodiment . 

In comparison with the user interface image of the 
previous embodiment, there is added an input area for 
registering and selecting the external environment of 
the system. In this image, a column with a downward 
arrow is provided in advance with plural selection 
items, which are displayed as a pull -down menu upon 
selection of the arrow mark. A desired item can be 
entered by selection in the pull-down menu. 

In the following there will be explained the 
functions of the present embodiment with reference to 
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Fig- 23 • When the user interface mode is accessed in 
the present system, a step s202 reads and displays the 
information of the user interface image shown in Fig. 
22. On this user interface image, the user enters the 
input device in an input device column, the display 
device in a display device column, the color 
temperature CCT^^ of the ambient illuminating light in 
an ambient lighting color temperature column, the 
luminance Y^s of the ambient illuminating light in an 
ambient light luminance column and the ambient lighting 
characteristic coefficient IHj,^ in an ambient lighting 
characteristic coefficient column. 

Otherwise the user enters, in an input column for 
the system external environment, the system environment 
data (input device, display device and image conversion 
matrix mentioned above) stored in advance by means to 
be explained later. 

The above-mentioned operations are selected 
according to discrimination in a step s208 whether the 
system external environment is selected. 

If the external environment is not selected, a 
step s203 enters the input device from the input device 
column, the display device from the display device 
column, the color temperature CCTj^^ of the ambient 
illuminating light from the ambient lighting color 
temperature column, and the luminance Y^^ of the ambient 
lighting from the ambient light luminance column and 
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the ambient lighting characteristic coefficient IKj^^ 
from the ambient lighting characteristic coefficient 
column. 

If the external environment is selected, a step 
5 s210 fetches the system environment data. 

Then a step s204 discriminates the selection of an 
execution button, and, if it is selected, the sequence 
proceeds to a step s205 to execute the operations of 
the flow chart. If the execution button is not 

10 selected, the sequence proceeds to a step s207 for 

discriminating the selection of an end button. If it 
is selected, the present operation mode is terminated. 
If the end button is not selected, the sequence 
proceeds to a step s208. A step s206 discriminates the 

15 completion of operations, and, if completed, the 

sequence proceeds to a step s209 . If not completed, 
the sequence waits in a stand-by state in the step 
s206. 

The step s209 discriminates whether the current 
20 external environment data of the system are to be 

stored, and, if not, the sequence returns to the step 
s208 . 

In case the data are to be stored, a name for 
storing the external environment is read from the user 
25 interface image, and the external environment data are 
stored under such name. 

The operations of the present system relating to 
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the user interface mode are executed in the 
above-explained manner. 

In the following there will be explained the 
operations in the execution of the user interface mode, 
5 with reference to Fig. 29, which is a flow chart 

showing the operations in the execution of the user 
interface mode in the present embodiment. 

The operations are basically same as those of the 
embodiment 4, except for the presence of a branched 
10 step in which the image conversion matrix is selected 
from the image conversion matrix storing unit 1113 and 
is set in the image conversion unit 1103. 

The system of the present embodiment, through the 
above explained operations and owing to the presence of 
15 the selection means by the user, the image conversion 
matrix calculating means and the storing means, allows 
to easily match the apparent colors of the object, the 
printed image and the image displayed on the display 
device under a given environment regardless of the 
20 external environment of the system such as the input 

device, display device and ambient illuminating light. 
[Embodiment 6] 

As in the embodiment 5, the present embodiment 6 
stores the image conversion matrix corresponding to the 
25 combination of the input device, display device and 
ambient lighting. 

Fig. 21 shows the color image display system of 
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the present embodiment, which is obtained by adding an 
image conversion matrix storing unit 1113 to the color 
image display system of the embodiment 4 shown in Fig. 
15. 

The difference between the present embodiment and 
the embodiment 5 lies in a fact whether the image 
conversion matrix to be employed is directly designated 
in the user interface unit by the external environment 
input column . 

In contrast to the embodiment 5 utilizing direct 
designation, the system control unit 1111 of the 
present embodiment is provided with a step of analyzing 
the input data of the user interface, and, if the 
result obtained in such analysis is same as a mark 
attached to the previously prepared and stored image 
conversion matrix, the previous matrix is selected 
without calculation of the matrix anew. 

In case the image conversion matrix is calculated 
anew in the present embodiment, the result of analysis 
of the user interface input data is displayed and 
stored in the image conversion matrix storing unit 
1113. 

The user interface image employed in the present 
embodiment is similar to that of the embodiment 1 shown 
in Fig. 17. 

The operations of the present embodiment are 
substantially similar to those of the foregoing 
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embodiment 5, except for the above-described 
operations • 

The system of the present embodiment, through the 
above-explained operations and owing to the presence of 
the image conversion matrix calculating means and the 
storing means, and the user interface input data 
analyzing means, allows to easily match the apparent 
colors of the object, the printed image and the image 
displayed on the display device under a given 
environment regardless of the external environment of 
the system such as the input device, display device and 
ambient illuminating light. 

As explained in the foregoing embodiments, in 
order to match the apparent colors in the display on 
the monitor and in the print, it is necessary to 
execute the conversion of the color signals in full 
consideration of the characteristics (spectral 
characteristics, color rendering property etc, ) of the 
ambient light. The foregoing embodiments are different 
in the method of setting the data required for such 
conversion, and the intention of the present invention 
is, in an image display method, an image display 
apparatus and an image display system, to provide means 
for setting the input device, display device and 
ambient illuminating light, and color signal conversion 
means capable, based on the characteristics of thus set 
input device, display device and ambient illuminating 
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light, of executing a color signal conversion based on 
the characteristics of the input device, a color signal 
conversion based on the characteristics of the ambient 
illuminating light, a color signal conversion including 
the color adaptation based on the characteristics of 
the ambient illuminating light and the displayed white 
color of the display device, and a color signal 
conversion based on the characteristics of the display 
device . 

More specifically, based on the information 
(chromaticity, color temperature or spectral intensity 
(luminosity)) on the illuminating light (ambient 
light), there are obtained information (chromaticity, 
XYZ stimulation values etc. ) on the white color sensed 
under such illuminating light and information (for 
example two-dimensional matrix) for converting other 
colors, and the color signal conversion is executed 
utilizing these information- 
According to the foregoing embodiments, there is 
achieved precise conversion of the color signals 
corresponding to various ambient light sources, thereby 
attaining sufficiently precise isochromaticity for the 
display on the monitor and for the printed image. 

Also the present invention is applicable to 
various hardware configurations and sequence processes 
corresponding thereto. Such sequence process may be 
realized into a logic circuit or a software or an 
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algorithm within the aforementioned scope of the 
present invention, and the hardware or the apparatus 
can be realized according to such algorithm. 
[Other embodiments] 

The present invention is applicable not only to a 
system consisting of plural equipment (for example host 
computer, interface equipment, reader, printer etc. ) 
but also to an apparatus consisting of a single 
equipment (for example a copying apparatus or a 
facsimile apparatus). 

The present invention also includes an embodiment 
of supplying a computer in an apparatus or a system, so 
connected with various devices as to realize the 
functions of the foregoing embodiments, with program 
codes of a software for realizing the functions of the 
foregoing embodiments whereby the effects of the 
present invention are achieved by the functions of such 
various devices according to the program stored in the 
computer of such system or apparatus. 

In such case, the program codes themselves of the 
above-mentioned software realize the functions of the 
foregoing embodiments, and the program codes themselves 
and means for supplying the computer with such program 
codes, for example a memory medium storing such program 
codes, constitute the present invention. 

The memory medium for storing the program codes 
can be, for example, a floppy disk, a hard disk, an 
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optical disk, a magnetooptical disk, a CD-ROM, a 
magnetic tape, a non-volatile memory card or a ROM. 

Also the present invention naturally includes not 
only a case in which the functions of the foregoing 
embodiments are realized by the execution of the 
supplied program codes by the computer but also a case 
in which the functions are realized by the cooperation 
of the program codes with the operating system of the 
computer or another application software. 

Furthermore the present invention includes a case 
in which the supplied program codes are stored in an 
expansion board of the computer or a memory provided ii 
an expansion unit connected to the computer and a CPU 
or the like provided in such expansion board or 
expansion unit executes all or a part of the actual 
process according to the instruction of such program 
codes thereby realizing the functions of the foregoing 
embodiments • 

The present invention is not limited to the 
foregoing embodiments but is subject to any and all 
modifications and variations within the scope and 
spirit of the appended claims. 
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WHAT IS CLAIMED IS: 

1. An image processing method for converting data 
dependent on a first illuminating light into data 
dependent on a second illuminating light, comprising 

5 steps of: 

storing conversion data for plural illuminating 
lights having different characteristics; 

generating data indicating the proportion of 
synthesis of said plural illuminating lights having 
10 different characteristics, corresponding to said second 
illuminating light; and 

converting data dependent on said first 
illuminating light into data dependent on said second 
illuminating light, based on said conversion data for 
15 plural illuminating lights having different 

characteristics, and said data indicating the 
proportion of synthesis. 

2. An image processing method according to claim 
20 1, wherein said plural illuminating lights are 

different in color rendering property. 

3. An image processing method according to claim 
1, wherein said data indicating the proportions of 

25 plural syntheses are stored in advance according to the 
kinds of the illuminating light- 
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4. An image processing met:hod according -to claim 
3, wherein the kind of said second illuminating light 
is designated by the user and said data indicating the 
proportion of synthesis are selected according to said 

5 designated kind of the second illuminating light. 

5. An image processing method according to claim 
1, wherein said data indicating the proportion of 
synthesis are generated according to a manual 

10 instruction of the user. 

6. An image processing method according to claim 
1, wherein said data indicating the proportion of 
synthesis are generated according to the output from a 

15 sensor for measuring the illuminating light. 

7. An image processing method according to claim 
1, wherein said conversion data are matrix data. 

20 8. An image processing apparatus for converting 

data dependent on a first illuminating light into data 
dependent on a second illuminating light, comprising: 

storage means for storing conversion data for 
plural illuminating lights having different 
25 characteristics; 

generation means for generating data indicating 
the proportion of synthesis of said plural illuminating 
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lights having different characteristics, corresponding 
to said second illuminating light; and 

conversion means for converting data dependent on 
said first illuminating light into data dependent on 
5 said second illuminating light, based on said 

conversion data for plural illuminating lights having 
different characteristics, and said data indicating the 
proportion of synthesis. 

10 9, A computer readable recording medium storing a 

program said program comprising the steps of: 

storing conversion data for plural illuminating 
lights having different characteristics; 

generating data indicating the proportion of 
15 synthesis of said plural illuminating lights having 

different characteristics, corresponding to said second 
illuminating light; and 

converting data dependent on said first 
illuminating light into data dependent on said second 
20 illuminating light, based on said conversion data for 
plural illuminating lights having different 
characteristics, and said data indicating the 
proportion of synthesis. 



25 

of: 



10. An image processing method comprising steps 
setting an ambient lighting characteristic 
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coefficient according to a manual instruction; 

inputting image data dependent on an input device; 

and 

effecting correction for the ambient lighting on 
5 said inputted image data based on said input device, a 
display device and said ambient lighting characteristic 
coefficient, thereby achieving conversion into image 
data dependent on said display device. 

10 11. An image processing method according to claim 

10, wherein said correction for the ambient lighting is 
achieved by correction of color rendering based on said 
ambient lighting characteristic coefficient. 

15 12. An image processing method according to claim 

11, wherein said correction of color rendering is 
achieved by a weighted process on said conversion data 
corresponding to the plural light sources having 
different color rendering properties, based on said 

20 ambient lighting characteristic coefficient. 

13 . An image processing method according to claim 
10, wherein matrix coefficients relating to said 
correction for the ambient lighting are calculated 
25 according to said ambient lighting characteristic 
coefficient. 
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14. An image processing method according to claim 
13, wherein said calculated matrix coefficients are 
registered according to a manual instruction. 

5 15. An image processing method according to claim 

10, further comprising a step of: 

setting the color temperature and the luminance of 
the ambient light; 

wherein said correction for the ambient light is 
10 achieved by a color adapted conversion according to 
said color temperature and luminance. 

16. An image processing apparatus comprising: 
setting means for setting an ambient lighting 

15 characteristic coefficient according to a manual 
instruction; 

input means for entering image data dependent on 
an input device; and 

conversion means for effecting correction for the 
20 ambient lighting on said entered image data based on 
said input device, a display device and said ambient 
lighting characteristic coefficient, thereby achieving 
conversion into image data dependent on said display 
device . 

25 

17. A computer readable recording medium storing 
a program for executing an image processing method. 
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said program comprising steps of: 

setting an ambient lighting characteristic 
coefficient according to a manual instruction; 

inputting image data dependent on an input device; 

and 

effecting correction for the ambient lighting on 
said inputted image data based on an input device, a 
display device and said ambient lighting characteristic 
coefficient, thereby achieving conversion into image 
data dependent on said display device. 
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ABSTRACT OF THE DISCLOSURE 

The invention is -fco obtain data under desired 
illumination with a limited memory capacity. 

There is provided an image processing method for 
5 converting data dependent on a first illuminating light 
into data dependent on a second illuminating light, the 
method comprises storing conversion data for plural 
illuminating lights of different characteristics, 
generating data indicating the proportion of synthesis 

10 of said plural illuminating lights of different 
characteristics, corresponding to the second 
illuminating light, and converting data dependent on 
the first illuminating light into data dependent on the 
second illuminating light, based on the conversion data 

15 for plural illuminating lights of different 

characteristics and the data indicating the proportion 
of synthesis. 
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zu 860) John A. O'Brien (Registration No. 24,367), John A. Krause (Registration No. 24,613), Henry J. Renk 
(Reoistration No. 25,499), Peter Saxon (Registration No. 24,947), Anthony M.Zupcic (Registration No 27,276) Charles 
P Baker (Registration No. 26,702), Stcvan J. Bosses (Registration No. 22 291), Edward E. Vassa lo (Registration No 
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vviiuaiH -vi. Wannisky (Registration ., ^ ^ ■ .. - too^o 

(Registration No. 30,051), Chnstopher Phihp Wnst (Registration No. 32,078), Gary M Jacobs (Registration No. 28,86 ), 
Kel K O'Neill Registration No. 32,622), Bruce C. Haas (Registration No. 32,734). Scott K. Reed (Registration No 
32,433). Scott D. Malpedc (Registration No. 32,533), Frednck M. Zullow (Registration No. 32,452), Richard P. Bauer 
(Registration No. 31,588), Warren E. Olsen (Registration No. 27,290), Abigail F. Cousins (Rcgisfration No. 29.292), 
Steven E. Warner (Registration No. 33,326), Thomas J. O'Connell (Registration No. 33,202), David L Schaeffer 
(Registration No. 32,716), Jack S. Cubert (Registration No. 24,245), Mark A. Williamson (Registration No. 33,628), Jean 
K Dudck (Registration No. 30,938), Raymond R. Mandra (Registration No. 34,382), Dominick A. Condc (Registration 
No 33 856), Pasquale A. Razzano (Reg. No. 25,512), John W. Behringcr (Registration No. 23,086) Robert C Kline 
(Reoistration No. 17,739), Mark J. Itri (Registration No. 36,171), Michael P. Sandonato (Registration No. 35,345), Jack 
M. Arnold (Registration No. 25,823), John D. Carlm (Registration No. 37,292), Darnel S. Glueck (Registration No. 
37,838), Joseph W. Ragusa (Registration No. 38,586), Bnan L. Klock (Registration No. 36,570), Anne M Maher 




Thomas Gellcnthien (Registration No. 39,683), Douglas Sharrott (Registration No 39,832) Gordon F- Sieckrnann 
(Reoistration Xo. 28,667), Jay H. Anderson (Registration No. 38,371), Nandu A. Talwalkar (Registration No. 41,339), 
Leonard J Santisi (Registration No. 24,135), Shu Muk Lee (Registration No. 41,147), Kathryn L. Sieburth (Registration 
Xo. 40,072), Lee A. Goldberg (Registration No. 38,894), and Lcisa M. Smith (Registration No. 39,378). 

Address all correspondence to: 
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COMBINED DECLARATION AND POWER OF ATTORNEY 
FOR PATENT APPLICATION 

(Page 2) 

FITZPATRICK, CELLA, HARPER & SCINTO 

30 Rockefeller Plaza 
New York, N.Y. 10112-3801 
Telephone No, (212) 218-2100 

I hereby declare that all statements made herein of my own knowledge are true and 
that all statements made on information and belief are believed to be true; and further that these 
statements were made with the knowledge that willful false statements and the like so made are 
punishable by fine or imprisonment, or both, under Section 1001 of Title 18 of the United States 
Code and that such willful false statements may jeopardize the validity of the application or any 
patent issued thereon. 

Full Name of Sole or First Inventor YOSHINQBU SHIRAIWA 

Inventor's signature 

Date Citizen/Subject of Japan 

Residence 17-23, Tamagawagakuen 3-chome. Machida-shi, Tokyo, Japan 

Post Office Address c/o CANON KABUSHIKI KAISHA. 3-30-2 Shimomaruko, Ohta-ku. 
Tokyo. Japan ^ . 

Full Name of Second Joint Inventor, if any TOSHIYUKI MIZUNO 

Second Inventor's signature , 

Date Citizen/Subject of Japan 

Residence 22-34, Kaminomiva 1-chome, Tsurumi-ku. Yokohama - shi , 
Kanagawa - ken , J apan . 

Post Office Address c/o CANON KABUSHIKI KAISHA, 3-30-2 Shimomaruko, 
Ohta-ku, Tokyo, Japan 

Full Name of Third Joint Inventor, if any YUMIKO HIDAKA 

Second Inventor's signature 

Date Citizen/Subject of Japan 

Residence 1-6, Higashi Naganuma, Inagi-shi, Tokyo. Japan 

Post Office Address c/o CANON KABUSHIKI KAISHA. 3-30-2 Shimomaruko, 
Ohta-ku, Tokyo, Japan 
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